Data owners, such as individuals and companies, are increasingly concerned that their private data, collected and shared by IT systems, is used only for the purposes for which it was collected. Additionally, entities collecting and managing this data must increasingly comply with regulations on how data is accessed and processed. Accordingly, there is a need to monitor IT systems to ensure that the data is being used correctly, and to identify and rectify incorrect behavior within an IT system.
Security information and event management (SIEM) tools provide a framework for collecting events from various components of an IT system and analyzing them to identify incorrect behavior. SIEM tools use data bases (DBs), data stream management systems (DSMSs), or other stream processing engines to analyze these streams of logged system events either online or offline, where they correlate events from different system components.
The inventors have recognized, however, that conventional SIEM tools are unable to efficiently monitor and identify incorrect behavior in many systems, particular systems where many thousands to millions of events occur each second.